We thank all reviewers for their time and effort in reviewing our paper. We set up experiments on PyTorch with ResNet18 (He et al., 2016) on CIFAR10 (Krizhevsky, 2009). Figure 1: Evaluations on CIFAR10: training loss ( 1 st column), test accuracy ( 2 nd column) and total number of transmitted bits. MB) on CIFAR10 are shown in Figure 1 above. We will release our code on GitHub in the final version.

Proving that "at convergence, the parameters of the approximate factors can be considered fixed" may We use the ML of the approximate GP in our work and also in EP . Reviewer 1 puts it well, that "replacing it [KL] by the L2 Wasserstein should strike the majority of researchers as an If the CDF is accessible, Equation (5), which forms the basis of our lookup tables, is stable and it avoids divergence. If the CDF isn't accessible, there are double integrations We thank reviewer 1 for their supportive comments and helpful suggestions on e.g. the broader impact, "Found analytically for fewer distributions than EP" While we agree Please see General Comment 1. "not clear if the proposed method is worth it" Please Please see General Comments 2.1 and 2.2. Besides, Figure 1.a and 1.b illustrate the effectiveness of our method in alleviating the over-estimation of variances "primary motivation and reason for pursuing QP over EP" Please see General Comment 3. "No analysis of fixed Approximate Inference, Opper & Winther] and proving the property pointed out by Reviewer 2 that "at convergence, However, we cannot yet rule out that our method is already provably convergent under appropriate assumptions. We will also use the extra page in the final version for this. "I'm not sure whether the page on the locality property is We ask the reviewer to kindly consider the broader relevance outlined in General Comment 3. "Note sure whether Please see General Comment 1. "The degree of novelty is pretty small" Please see General Comment 3. "marginal likelihood and its accuracy" Please see General Comments 2.1 and 2.2. Please see General Comment 4. "A discussion why values for p Appendix B. These cases are interesting but also even more challenging to handle.

We would like to thank the reviewers for their time and effort to read our paper and provide constructive suggestions. We carefully addressed all comments as closely as possible. Thus, please give a chance to publish our work in NeurIPS 2020. According to your instruction, we will move the core definitions in Appendix to the main paper. Please appreciate in-depth and theoretical analysis of the proposed method in the main paper and Appendix.

We now turn to address each of the reviewers individual comments. We do not share your feelings regarding the claim that "the potential audience in the NeurIPS community is limited". Regarding your comment "presentation is unusually technical for machine learning venues", we would like to point out We believe the example in Table 2 demonstrates exactly this quite nicely. Reviewer #2: Thank you for you positive feedback and for for finding our results significant. We will add an appropriate discussion to make this point clearer.

We thank all reviewers for their careful reading of the paper, thoughtful feedback, and constructive suggestions. Each reviewer's major comments are addressed below. Distinct novelties relative to Ref. [31] are: i) Algorithm: The present submission develops Following your suggestion, [31] will be discussed more thoroughly in the revised paper. We will respectfully disagree that it "makes more sense to take a decaying step-size." Due to space limitation, the focus of this paper was placed on analysis under both IID and Markovian data .

We will improve the overall presentation and add more details for better accessibility. Thank you for your review and clarifying questions. For interpretability, we chose the bank dataset (c.f. Feature set description is provided in Appendix C.1. Circles in Figure 2(c) signify that Katz centrality for those agents was increased when perturbing structure for testing.

We thank all the reviewers for their time and effort to evaluate our paper. We will discuss the assumptions in the context of the SDE in Eq. 4, which is our main The boundedness assumption requires the solutions to the SDE to be'non-explosive' in the sense that for On the other hand, in Thm.S3, we already proved a bound which does not require H4, but This assumption is common in statistics (see [Bra83]) but hard to verify in practice. We agree that Prop.1 can be difficult to grasp at a first sight; hence, we provided BN is not fully understood. Our main focus is the relationship between generalization and intrinsic dimensionality. Relationship between BN and dimensionality is an interesting future direction which would complement our work.

We would like to thank the reviewers for their comments and feedback. Janzing et al. [9] write down the same equation, but We will follow the reviewer's The decomposition for conditional SVs follows by replacing "conditioning The decomposition is introduced in Section 3 to assist our illustration of how the different SVs attribute a model's SVs. Unlike conditional (asymmetric) SVs, causal SVs provide the right intuition in the case of common confounding. See also the previous paragraph. SVs appear to fare better than the reviewer suggests.

Please see response 3 under reviewer 4. Final-version plots will be made printer friendly.

Large language models have recently demonstrated advanced capabilities in solving IMO and Putnam problems; yet their role in research mathematics has remained fairly limited. The key difficulty is verification: suggested proofs may look plausible, but cannot be trusted without rigorous checking. We present a framework, called LLM+CAS, and an associated tool, O-Forge, that couples frontier LLMs with a computer algebra systems (CAS) in an In-Context Symbolic Feedback loop to produce proofs that are both creative and symbolically verified. Our focus is on asymptotic inequalities, a topic that often involves difficult proofs and appropriate decomposition of the domain into the "right" subdomains. Many mathematicians, including Terry Tao, have suggested that using AI tools to find the right decompositions can be very useful for research-level asymptotic analysis. In this paper, we show that our framework LLM+CAS turns out to be remarkably effective at proposing such decompositions via a combination of a frontier LLM and a CAS. More precisely, we use an LLM to suggest domain decomposition, and a CAS (such as Mathematica) that provides a verification of each piece axiomatically. Using this loop, we answer a question posed by Terence Tao: whether LLMs coupled with a verifier can be used to help prove intricate asymptotic inequalities. More broadly, we show how AI can move beyond contest math towards research-level tools for professional mathematicians.